-
Notifications
You must be signed in to change notification settings - Fork 206
Android SDK build scripts #467
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
* Build Android image * Checkout without ssh * Retry build if it fails * Swift 6.1 Release Dockerfiles (swiftlang#456) * Change binutils-gold package dependency on Debian 12 to binutils (swiftlang#457) * Update installed packages after nightly platform expansion (swiftlang#458) * update nightly-6.1 dependencies * update nightly-main dependencies * fix ubuntu images * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Fedora 41 Dockerfile (swiftlang#464) * Build Android image * Build Android image * Swift 6.1 Release Dockerfiles (swiftlang#456) * Change binutils-gold package dependency on Debian 12 to binutils (swiftlang#457) * Build Android image * Build Android image * Build Android image * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Android SDK build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Androd build * Swift Android build * Swift Android build * Swift Android build * Swift Android build --------- Co-authored-by: Mishal Shah <[email protected]> Co-authored-by: Chris McGee <[email protected]> Co-authored-by: Justice Adams <[email protected]> Co-authored-by: Andrew Sukach <[email protected]>
* Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Start splitting NDK out from the rest of the SDK * Start splitting NDK out from the rest of the SDK * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2 * Swift Android build 6.2
* Swift Android build 6.2 * Swift Android build 6.2 * Add post-install script to SDK bundle * Add post-install script to SDK bundle * Add post-install script to SDK bundle * Add post-install script to SDK bundle
I've updated this PR to build the SDK without including the Android NDK. In order to accommodate this, there is now a post-install The latest result of the build for the 6.2 nightly can be seen at: https://github.com/skiptools/swift-android-toolchain/releases/tag/6.2-DEVELOPMENT-SNAPSHOT-2025-05-07-a |
To clarify what the post-install script is doing, following is a result of the output of the command with the default NDK directory:
Alternative NDK locations can be specified by overriding the
|
* Add static libraries to post-install script * Add post-install script to SDK bundle * Add post-install script to SDK bundle * Update submodules
As of 72964f5, the Android SDK is now being built in a Docker container. |
…z,max-page-size=16384
* Address nits * Eliminate unnecessaryt pushds and use run cmake install instead of ninja * Generate SBOM with SDK dependencies * Add example of command for building locally
…ubstitutions instead.
@etcwilde @shahmishal Now that this PR is ready to go, can you let us know any final steps we need in order to integrate this with the Swift snapshot/release process? Are the scripts for the static Linux and WASM SDKs run as part of the internal CI, or are they performed manually by someone? I also notice that @al45tair recently added some command-line flags (in #476) to the static Linux SDK build like |
@marcprux The Static SDK for Linux gets built by a separate Jenkins job, e.g. for 6.2: https://ci.swift.org/view/Swift%206.2/job/oss-swift-6.2-package-static-sdk/ It isn't built manually, except when working on it locally. |
Thanks for the insight! So all we need is an equivalent "Android SDK" CI like the Static SDK job. Is the source script for this job available for inspection somewhere? Is this something I can help develop and/or contribute? @etcwilde mentioned at the Android Workgroup meeting on Tuesday that he would do a final review of this PR, so I expect this should be merged imminently, and then the build script will be ready. As I've mentioned, we're really itching to start getting Android SDK snapshots published so we have time for some real-world testing before a final 6.2 release. We're already building the SDK periodically in an unofficial GitHub action (e.g., see the latest artifacts at https://github.com/swift-android-sdk/swift-android-sdk-build/actions/runs/17320318144), so all that remains is to get this into Jenkins and then add whatever deployment step is necessary to get it published to swift.org. As I say, I'm happy to help out with any tasks needed to get this over the finish line. Just let me know what I can do. CC: @shahmishal |
Can we get some attention on this PR? Even if you don't start running it right away as part of the Swift release process, can we at least get it merged so it is ready to go? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this looks good. Thank you for working through it.
We should resolve this failure before merging the change:
|
How are you running it? Manually? The build action in the GitHub CI is passing:
Are you running it the
|
We have script ./ci_test.py in the repo which is ran by the GitHub Actions. |
I'm unclear on the expected environment for this command. As this was modeled on the static-linux Dockerfile, I based the android SDK Dockerfile on the assumption that the file would be built with the working directory being the Static Linux Docker error
As a test, I tried making a PR change to the static-linux SDK's Dockerfile at marcprux#1, and the CI also errored out when running I'm happy to fix anything that needs fixing, I'm just not sure what the expectation is for the environment and working directory of the |
Also, 62c8581 seems to indicate that it is intended that the Docker build will be run from the folder in which it resides:
|
.github/workflows/pull_request.yml
Outdated
echo 'import Dispatch' >> Sources/DemoProject/DemoProject.swift | ||
echo 'import Android' >> Sources/DemoProject/DemoProject.swift | ||
- name: Test Demo Project on Android | ||
uses: skiptools/swift-android-action@main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are we using this action for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing, this entire github workflow change is just a demo. We're looking for guidance on how you want to do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we discussed in the review for the file, that whole android-build
section of the workflow is just for validating the build process while we are iterating on the PR. It has been very useful for ensuring that all the Android SDK build variants run successfully, but we figured we might not want it in the final PR workflow.
I've excised that whole section in 8eb982e.
.github/workflows/pull_request.yml
Outdated
repository: apple/swift-algorithms | ||
path: swift-algorithms | ||
- name: Test swift-algorithms | ||
uses: skiptools/swift-android-action@main |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are we using this action for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also removed in 8eb982e
# use an out-of-tree build folder | ||
export SWIFT_BUILD_ROOT=${build_dir}/swift-project | ||
|
||
./swift/utils/build-script \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a blocker: We should create a preset in the swift repo, incase someone changes the flags we don't break android sdk.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My goal is to eventually obsolete this build script by replacing it entirely with a build preset and a call to the swift-sdk-generator
tool. However, since we couldn't get that done before the 6.2 release, we chose to instead reuse the static linux SDK build script as much as possible, which you can see by just running diff -ruw
between this new Android build and the current static linux SDK build.
The only differences to the static linux SDK build are that we use the official build-script
to build much more of the toolchain, had to add a post-install script because configuring external C/C++ platform SDKs like the Android NDK is still partially broken in SwiftPM, swiftlang/swift-package-manager#8958, (the static linux SDK doesn't face this problem because it ships Musl libc and libc++ in the bundle, which we didn't want to do with the C/C++ libraries from the Android NDK), and instead of the single build
script, we ship both a build-docker
and a build-local
, depending on whether you want to use Docker or not (we know you only use Docker on this official CI, but we provided it for the convenience of others who don't want Docker).
We need to add license header to the new files.
|
Thanks for looking into this. We can file a GitHub Issue for now, and resolve this in a future PR. |
Btw, Mishal, we have designed this for two separate purposes, as you asked for:
Just let us know what other build options you want and what interface you'd like for controlling those options and we can add it. Also, this currently builds and tests from trunk and the release/6.2 branch without a hitch, but my last pulls for 6.2.1 and 6.2.0 are still pending approval, so we will need to get those in or apply them externally to produce 6.2.1 or 6.2.0 Android bundles with this Docker build. |
@shahmishal I am guessing that the next step will be to get this build running in the CI, similar to https://ci.swift.org/job/oss-swift-6.2-package-static-sdk/. Is there anything we can do to help move that along? We'd love to start seeing the builds running along on their own so we can swap out the GitHub and community CI build jobs, and update the status grid at https://github.com/swiftlang/swift?tab=readme-ov-file#swift-programming-language |
Following up on swiftlang/github-workflows#106 and swiftlang/swift#80788, this PR contains a new
swift-docker/swift-ci/sdks/android/
folder with scripts that will build and upload a full stand-alone Swift Android Swift SDK. It is modeled after the structure of the swift-ci/sdks/static-linux build scripts.Running
The top-level
./build
script installs a host toolchain and the Android NDK, and then invokesscripts/fetch-source.sh
which will fetch tagged sources for libxml2, curl, boringssl, and swift.It then applies some patches and invokes
scripts/build.sh
, which will build the sources for each of the specified architectures. Finally, it combines the NDK and the newly built SDKs into a single artifactbundle.Specifying Architectures
By default all the supported Android architectures (
aarch64
,x86_64
,aarmv7
) will be built, but this can be reduced in order to speed up the build. This can be useful, e.g., as part of a CI that validates a pull request, as building a single architecture takes around 30 minutes on a standard ubuntu-24.04 GitHub runner, whereas building for all the architectures takes over an hour.To build an artifactbundle for just the
x86_64
architecture, run:Installing and validating the SDK
The
.github/workflows/pull_request.yml
workflow will create and upload an installable SDK named something like:swift-6.1-RELEASE_android-0.1.artifactbundle.tar.gz
. The results of one of the workflow runs can be seen at https://github.com/swift-android-sdk/swift-docker/actions/runs/14603885089The workflow will also install the SDK locally and use swift-android-action to build and test various Swift packages in an Android emulator.